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Abstract 


Telepresence operations require high quality information transfer between the 
human master and the remotely located slave . Present Air Force research focusses 
on the human aspects of the information needed to complete the control/ feedback 
loop . Work in three key areas of human sensory feedback for manipulation of 
objects are described . Specific projects in each key area are outlined, 
including research tools (hardware ) , planned research, and test results . Non - 
manipulative feedback technologies are mentioned to complete the advanced 
teleoperation discussions • 


Introduction 


The Air Force renewed its interest in 
remotely operated manipulator systems 
in 1985 with Project Forecast II. 
The current concept is based on the 
idea of a high quality teleoperated 
master-slave manipulator system [ 1 ) — 
[2]. The present program focus is 
centered on the feedback signals that 
must be generated to adequately dis- 
play force information to the human 
operator. 

The key to the successful fielding of 
teleoperator systems is the design 
and implementation of a good human- 
machine interface. Three human sens- 
es (touching, hearing, and seeing) 
hold the highest potential for effec- 
tive human sensory feedback. While 
each sense is somewhat independent, 
the senses also complement each other 
in the human though the interdepen- 
dence is not well understood. Humans 
use visual feedback to gather large 
amounts of high frequency, high reso- 
lution information about the environ- 
ment. Likewise, a telepresence sys- 
tem can present significant quanti- 
ties of information via visual dis- 
plays. However, the visual system is 
ineffective in some environments such 
as heavy smoke, darkness or situa- 
tions with obstructed vision. In 
these cases, humans often compensate 
with audio or tactile feedback. 
Audio feedback can provide informa- 
tion that is essentially omnidirec- 
tional and invisible. Touch and 
proprioceptive feedback can provide 
critical information regarding manip- 
ulative tasks. Tactile feedback can 


supplement visual feedback and in 
many cases permit completion of the 
task without vision. 

Remote Manipulation Spectrum 

Prior to discussing the details of 
the human sensory feedback ( HSF ) , it 
is important to identify the areas 
within the teleoperation spectrum 
where HSF is important. Sheridan [3] 
offers an excellent review of the 
subject and provides well-ordered 
definitions to each key area. Sheri- 
dan's definitions are included to 
provide a common point of departure: 

" Telepresence is the ideal of sensing 
sufficient information about the 
teleoperator and task environment, 
and communicating this to the human 
operator in a sufficiently natural 
way, that the operator feels physi- 
cally at the remote site.** 

"Teleoperation is the extension of a 
person's sensing and manipulation 
capability to a remote location. A 
teleoperator includes at the minimum 
artificial sensors, arms and hands, a 
vehicle for carrying these, and com- 
munication channels to and from the 
human operator. The term "tele- 
operation" refers most commonly to 
direct and continuous human control 
of the teleoperator, but can also be 
used generally to encompass "tele- 
robotics" (see below) as well." 

" Telerobotics is a form of tele- 
operation in which a human operator 
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acts as a supervisor, intermittently 
communicating to a computer about 
goals, constraints, plans, contingen- 
cies, assumptions, suggestions, and 
orders relative to a limited task, 
getting back information about accom- 
plishments, difficulties, concerns, 
and, as requested, raw sensory data — 
while the subordinate telerobot exe- 
cutes the task based on information 
received from the human operator plus 
its own artificial sensing and intel- 
ligence. " 

" Supervisory Control in the present 
context is mostly synonymous with 
telerobotics, referring to the analo- 
gy of a human supervisor directing 
and monitoring the activities of a 
human subordinate. The term "super- 
visory control" is used commonly to 
refer to human supervision of any 
semi-autonomous system (including an 
aircraft, a chemical or power plant, 
etc.), while 'telerobot' commonly 
refers to a device having arms for 
manipulating or processing discrete 
objects in its environment." 

"Robotics is the science and art of 
performing, by means of an automatic 
apparatus or device, functions ordi- 
narily ascribed to human beings, or 
operating with what appears to be 
almost human intelligence (adapted 
from Webster's Third International 
Dictionary ) . " 


As can be seen in Figure 1, the level 
of human operator involvement varies 
with each key area. 



Armstrong Laboratory Research 

Human Sensory Feedback Research at 
the Armstrong Laboratory's Crew Sys- 
tems directorate is centered in three 
areAs of the force feedback domain: 

Coarse Positioning - Movement pro- 
duced by large scale motion associat- 
ed with the human arm. Coarse manip- 
ulation implies movement to position 
a dexterous end effector in the prop- 
er orientation to perform work via 
fine manipulation. 

Fine Manipulation - Small scale mo- 
tion associated with the human hand. 
Fine manipulation is the performance 
of tasks using highly dexterous end 
effectors. 

Tactile Feedback - Determine the role 
of tactile feedback from the remote 
system to the human operator. Iden- 
tify and investigate technologies 
important to providing high fidelity 
tactile feedback to human operators. 


Coarse Positioning Research 

Current coarse positioning investiga- 
tions are evaluating the impact of 
exoskeletal systems on human perfor- 
mance. The method is to measure 
human performance using an instru- 
mented "peg-in-the-hole ' task board 
based on Fitts' Law [4] [5]. The 
unencumbered operator executes a test 
sequence to obtain a baseline of task 
performance. The operator then dons 
a non-force-reflecting (NFR) exoskel- 
eton and re-executes the test se- 
quence. The difference yields an 
indicator of the restriction of the 
exoskeleton on the human's task per- 
formance. Results of the initial 
study show a 32 to 41 percent reduc- 
tion in task performance. This reduc- 
tion in task performance is due sole- 
ly to the interference caused by the 
NFR exoskeleton. 

Follow-on testing will evaluate task 
performance as the human operator 
uses the exoskeleton to teleoperate a 
slave robot. The result will be a 
system-dependent indicator of the 
impact of the total master-slave 
system on the task performance. The 
preliminary testing was done with a 
NFR exoskeleton. The final sequence 
in this phase will completed with a 
NFR exoskeleton. This NFR exoskele- 
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ton encumbrance test forms the basis 
for subsequent testing using the 
force reflecting exoskeleton de- 
scribed below. 

For the past two years , Odetics, 
Inc., has been developing a Force 
REFLecting EXoskeleton (FREFLEX) to 
evaluate the impact on task perfor- 
mance of force reflection with a non- 
constraining seven-degree-of-freedom 
force-reflecting exoskeleton. Design 
of the cable-driven system is dis- 
cussed in [6]. 

A similar series of test sequences 
will be conducted using the FREFLEX. 
The encumbrance of the FREFLEX will 
be measured in the NFR mode as be- 
fore. Once the passive baseline is 
determined, a series of force re- 
flecting experiments will be conduct- 
ed to evaluate several parameters 
such as gravity and friction compen- 
sation, feedforward and feedback 
control algorithms, and modified 
kinematic and dynamic parameters. 
The expected result is an increased 
understanding of each of the factors 
that impact the human operator in a 
complex telepresence system. The end 
goal is to develop design parameter 
guidelines that are based on knowl- 
edge of the human characteristics in 
the telepresence system. 


A second interest area in coarse 
positioning is to quantify kinematic 
and dynamic limitations placed on the 
operator by the exoskeleton system. 
Research in this area is centered on 
the WAT SMART™ [ 14 ] , a three dimension- 
al motion analysis system. The ap- 
proach is to measure the operator's 
kinematic and dynamic parameters with 
and without the exoskeleton. The 
objective is to examine the joint-by- 
joint kinematic and dynamic encum- 
brances placed on the operator by the 
exoskeleton. The resultant data is 
expected to lead to human-based para- 
metric design guidelines for exoskel- 
etons. 

The WAT SMART™ data will also be used 
to establish velocity and accelera- 
tion profiles of each joint. In 
addition to evaluating the encumbran- 
ce effects, these velocity and accel- 
eration profiles are necessary to 
develop mechanical performance param- 
eters of motors and actuators for 
future exoskeleton devices. 


Fine Manipulation Research 

Fine manipulation research at the 
Armstrong Laboratory is centered on 
the pair of dexterous hands designed 
by Jacobson at the University of Utah 
[7]. Initial plans were to combine 
the robotic hands with the robotic 
arms to form a slave system for tele- 
presence research. Funding reduc- 
tions have prevented arm-hand inte- 
gration but theoretical research has 
continued. Whalen [8] developed a 
basis for planar grasping as a foun- 
dation for dexterous robotic hands. 
Future work will be directed toward 
understanding the relationship of 
each digit of the robotic hand. 

A key component to successful fine 
manipulation via telepresence is the 
development of kinematic and dynamic 
models between the slave hand, the 
operator's hand-master controller, 
and the operator's hand. Initial 
kinematic mapping from the slave hand 
to the master hand controller to the 
human operator's hand has been com- 
pleted [9]. Force feedback for 

hand-master controllers will require 
innovative actuation schemes. Ini- 
tial thoughts on hand master control- 
lers with feedback indicate require- 
ments for high speed, lightweight 
actuators with high power densities 
and approximately linear responses. 
Current plans call for feedback sig- 
nals for the acutators to be derived 
from tendon tension signals on the 
dexterous hands. Long term objec- 
tives will combine the hand master 
control loop with the slave hand 
control loop to produce a force feed- 
back capability in fine manipulation. 
Auxiliary force information from 
tactile sensors on the hands may also 
be integrated into the overall feed- 
back scheme to provide even more 
stable control. 

Tactile Feedback Research 

The third, but much less understood, 
area of force feedback is tactile 
feedback. Tactile feedback plays a 
critical role in human performance. 
Armstrong Laboratory work in this 
area has two key thrusts: (1) The 

addition of sensors to one of the 
dexterous hands (2) the development 
of small lightweight tactile stimula- 
tors . 

A Phase II Small Business Innovation 


118 



Research (SBIR) project will yield a 
sensorized robotic hand in FY93. The 
sensor suite includes palmar sensors 
as well as sensors on the back of the 
robotic fingers. Additional sensor 
research is ongoing at the Air Force 
Institute of Technology. Nering [10] 
is developing a slip sensor using 
resistive paint and artificial neural 
networks. Other AFIT researchers 
[11] [12] have explored the use of 
polyvinlylidene fluoride film to 
fabricate high-density contact posi- 
tion and force sensors on silicon 
substrate . 

The second component of the tactile 
feedback subsystem is the tactile 
stimulator for the human operator's 
hand. While a specific technology 
has not been selected, present work 
focusses on Shape Memory Alloy (SMA) . 

The current device is a five-by-six 
(30-element) array using SMA actua- 
tors for stimulator pins [13]. 

Psychophysical perception testing on 
human subjects is planned to evaluate 
the SMA applicability to tactile 
stimulator technology. Follow-on 
investigations will explore operator 
capability to detect active and stat- 
ic patterns and to identify dynamic 
patterns. Long term plans call for 
the integration of the robotic tac- 
tile sensors with the operator tac- 
tile stimulators to achieve a closed 
loop feedback subsystem in the tac- 
tile domain. 

Conclusion: 

Armstrong Laboratory research in 
human-in-the-loop controlled robots 
is focusing in three key areas. 
Coarse positioning, using force-re- 
flecting exoskeletons, is the key 
element in positioning the robotic 
end effector to do work. Once the 
coarse positioning subsystem has the 
manipulator in place, the fine manip- 
ulation task will be performed by 
highly dexterous robotic hands. 
Force feedback to the human operator 
via a force-reflecting hand master is 
planned. Tactile feedback research 
centers on shape memory alloy tech- 
nology for tactile stimulators. The 
stimulators will be combined with the 
robotic tactile sensor suites to 
yield a complete tactile subsystem. 

Long term plans call for integration 
of the three manipulation areas into 


a single manipulation system with 
intuitive force feedback. It is 
anticipated that video feedback sub- 
systems and audio subsystems will 
have off-the-shelf availability when 
the manipulation matures sufficiently 
for total system integration. 
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